A new class of entropy estimators for multi-dimensional densities
نویسنده
چکیده
We present a new class of estimators for approximating the entropy of multi-dimensional probability densities based on a sample of the density. These estimators extend the classic ”m-spacing” estimators of Vasicek and others for estimating entropies of one-dimensional probability densities. Unlike plug-in estimators of entropy, which £rst estimate a probability density and then compute its entropy, our estimators avoid the dif£cult intermediate step of density estimation. For £xed dimension, the estimators are polynomial in the sample size. Similarities to consistent and asymptotically ef£cient one-dimensional estimators of entropy suggest that our estimators may share these properties.
منابع مشابه
A New Class of Decentralized Interaction Estimators for Load Frequency Control in Multi-Area Power Systems
Load Frequency Control (LFC) has received considerable attention during last decades. This paper proposes a new method for designing decentralized interaction estimators for interconnected large-scale systems and utilizes it to multi-area power systems. For each local area, a local estimator is designed to estimate the interactions of this area using only the local output measurements. In fact,...
متن کاملExponential Concentration of a Density Functional Estimator
We analyze a plug-in estimator for a large class of integral functionals of one or more continuous probability densities. This class includes important families of entropy, divergence, mutual information, and their conditional versions. For densities on the d-dimensional unit cube [0, 1] that lie in a β-Hölder smoothness class, we prove our estimator converges at the rate O ( n− β β+d ) . Furth...
متن کاملConcentration Inequalities for Density Functionals
Estimating various kinds of entropy, mutual information, divergence, and other kinds of integral functionals of probability densities in a statistically consistent manner is of great importance in many machine learning tasks. Although this is a fundamental problem in nonparametric statistics, to the best of our knowledge there have been no finite sample exponential concentration bounds derived ...
متن کاملEstimation of Lower Bounded Scale Parameter of Rescaled F-distribution under Entropy Loss Function
We consider the problem of estimating the scale parameter &beta of a rescaled F-distribution when &beta has a lower bounded constraint of the form &beta&gea, under the entropy loss function. An admissible minimax estimator of the scale parameter &beta, which is the pointwise limit of a sequence of Bayes estimators, is given. Also in the class of truncated linear estimators, the admissible estim...
متن کاملEmpirical estimation of entropy functionals with confidence
Nonparametric estimation of functionals of density from finite number of samples is an important tool in domains such as statistics, signal processing and machine learning. While several estimators have been proposed in literature, the performance of these estimators is not known. We propose a kNN class of plug-in estimators for estimating non-linear functionals of density, such as entropy, mut...
متن کامل